Method for stitching image data captured by multiple vehicular cameras

ABSTRACT

A method for stitching image data captured by multiple vehicular cameras includes equipping a vehicle with a vehicular vision system having a control and a plurality of cameras disposed at the vehicle so as to have respective fields of view exterior the vehicle. Image data captured by first and second cameras of the plurality of cameras is processed to detect and track an object present in and moving within an overlapping portion of the fields of view of the first and second cameras. Image data captured by the first and second cameras is stitched, via processing provided captured image data, to form stitched images. Stitching of captured image data is adjusted responsive to determination of a difference between a feature of a detected and tracked object as captured by the first camera and the feature of the detected and tracked object as captured by the second camera.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 16/693,515, filed Nov. 25, 2019, now U.S. Pat. No. 10,780,827,which is a continuation of U.S. patent application Ser. No. 16/246,624,filed Jan. 14, 2019, now U.S. Pat. No. 10,486,596, which is acontinuation of U.S. patent application Ser. No. 14/191,512, filed Feb.27, 2014, now U.S. Pat. No. 10,179,543, which claims the filing benefitsof U.S. provisional applications, Ser. No. 61/793,592, filed Mar. 15,2013, and Ser. No. 61/770,051, filed Feb. 27, 2013, which are herebyincorporated herein by reference in their entireties.

FIELD OF THE INVENTION

The present invention relates generally to vision systems for vehiclesand, more particularly, to vision systems having a plurality ofexteriorly facing cameras disposed at a vehicle.

BACKGROUND OF THE INVENTION

Rear backup cameras and other exterior vision systems are known for usein vehicles. Examples of such systems are described in U.S. Pat. Nos.7,859,565; 6,611,202; 6,222,447; 5,949,331; 5,670,935 and/or 5,550,677,which are hereby incorporated herein by reference in their entireties.Such systems may display images for viewing by the driver of the vehiclethat provide a view exterior of the vehicle. It is known to provide aplurality of cameras at a vehicle, such as a forward facing camera, arearward facing camera and opposite sideward facing cameras, and tostitch together images captured by the cameras to provide a surroundview or top down view for displaying for viewing by a driver of thevehicle.

SUMMARY OF THE INVENTION

The present invention provides a means for dynamically calibrating oradjusting the image stitching of the images captured by two or morecameras of a multi-camera vision system of a vehicle, such as when theload or level of the vehicle changes.

The present invention provides a dynamic image stitching system orprocess that monitors overlapping regions of fields of view of adjacentcameras (such as a side camera and a front or rear camera) and adjuststhe image stitching of the captured images to adapt the image stitchingwhen the load of the vehicle changes or when the road level or terrainchanges or the like. A display screen displays the stitchedsubstantially seamless images for viewing by a driver of the vehiclewhen the driver is normally operating the vehicle.

Optionally, the dynamic image stitching system of the present inventionmay include an image processor that is operable to adjust a stitchingalgorithm responsive to a determination of a change in at least onecharacteristic of a feature or object present in an overlapping regionof the fields of view of two cameras at the vehicle (such as a sidecamera and a rear camera or such as a side camera and a front camera orthe like). For example, the processor may determine a feature or objectpresent in the overlapping region of the cameras' fields of view, andthe processor may compare a characteristic of the feature or object inimage data captured by a first camera with the correspondingcharacteristic of the feature or object in image data captured by asecond camera. The processor adjusts the stitching algorithm responsiveto a determination of a difference between a characteristic of a featureor object in image data captured by the first camera and thecharacteristic of the feature or object in image data captured by thesecond camera. For example, the processor may process captured imagedata to determine whether the feature as captured by the first camera ismisaligned with the feature as captured by the second camera, and theprocessor may adjust the stitching algorithm responsive to such adetermination.

These and other objects, advantages, purposes and features of thepresent invention will become apparent upon review of the followingspecification in conjunction with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a plan view of a vehicle equipped with a multi-camera visionsystem in accordance with the present invention;

FIG. 2 is a top down view of a vehicle, with arrows indicating thegeneral fields of view of the cameras at the vehicle;

FIG. 3 shows the images captured by the cameras of the vehicle of FIG.2;

FIG. 4 is a plan view of a vehicle disposed at a calibration pattern;

FIG. 5 is an image captured by a side camera of a vehicle equipped withthe vision system of the present invention;

FIG. 6 is an image captured by a rear camera of a vehicle equipped withthe vision system of the present invention;

FIG. 7A is a schematic showing a vehicle with an ultrasound distancemeasuring system having an ultrasound sensor integrated into a sidemirror housing of the vehicle in accordance with the present invention,shown with the vehicle in a horizontal level;

FIG. 7B shows a vehicle similar to FIG. 7A, but with the vehicle rolledto its right by an angle ψ, with the measuring system that rolls withthe vehicle detecting a closer distance to objects and the ground;

FIG. 8 shows the image portions taken by the respective cameras, whichadd to a stitched virtual top view image, shown with objects symbolizedby boxes passing the scene surrounding the vehicle when the vehicle isdriving forward;

FIG. 9 shows a stitched virtual top view image derived from the imageportions, when the vehicle is leveled normally the objects are supposedto pass the scene in generally straight lines when the vehicle isdriving straight, shown with a picked out test object highlighted inblack;

FIG. 10 shows the identical scene as shown in FIG. 9, with two testobjects highlighted and other objects erased (for clearer illustration),showing that, when the vehicle is leveled normally, the test objects aresupposed to pass the scene in generally straight lines sideward or leftand right from the vehicle when the vehicle is driving straight;

FIG. 11 shows a scene similar to these in FIGS. 9 and 10, with two testobjects highlighted, with the vehicle driving straight as in FIGS. 9 and10, but with the vehicle rolled by an angle ψ to the right, when therolling angle ψ is uncompensated by the vision system, the passing pathsof the tracked (and also the untracked) objects do not follow straightlines and the stitching thresholds do not match any more;

FIGS. 12-14 are perspective views of a parking maneuvering situation(partially on the sidewalk), showing overlays in accordance with thepresent invention; and

FIGS. 15-17 are perspective views of a highway situation, showingoverlays in a view such as may be shown in a rearview side mirrorreplacement camera display in accordance with the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring now to the drawings and the illustrative embodiments depictedtherein, a vehicle 10 includes an imaging system or vision system 12that includes at least one exterior facing imaging sensor or camera,such as a rearward facing imaging sensor or camera 14 a (and the systemmay optionally include multiple exterior facing imaging sensors orcameras, such as a forwardly facing camera 14 b at the front (or at thewindshield) of the vehicle, and a sidewardly/rearwardly facing camera 14c, 14 d at respective sides of the vehicle), which captures imagesexterior of the vehicle, with the camera having a lens for focusingimages at or onto an imaging array or imaging plane or imager of thecamera (FIG. 1). The vision system 12 includes a control or processor 18that is operable to process image data captured by the cameras and mayprovide displayed images at a display device 16 for viewing by thedriver of the vehicle (although shown in FIG. 1 as being part of orincorporated in or at an interior rearview mirror assembly 20 of thevehicle, the control and/or the display device may be disposed elsewhereat or in the vehicle).

The dynamic top view vision system of the present invention stitchescaptured images to form a composite substantially seamless image, andmay utilize aspects of the vision systems described in U.S. Pat. Nos.6,222,447; 5,949,331; 5,760,962; 5,670,935 and/or 5,550,677, and/or U.S.patent applications, Ser. No. 12/508,840, filed Jul. 24, 2009, whichpublished on Jan. 28, 2010 as U.S. Publication No. US-2010-0020170; Ser.No. 13/848,796, filed Mar. 22, 2013, and published on Oct. 24, 2013 asU.S. Publication No. US-2013-0278769, Ser. No. 13/852,190, filed Mar.28, 2013, and published on Aug. 29, 2013 as U.S. Publication No.US-2013-0222593, Ser. No. 13/894,870, filed May 15, 2013, now U.S. Pat.No. 10,089,537, and/or Ser. No. 14/046,174, filed Oct. 4, 2013, now U.S.Pat. No. 9,723,272, which are hereby incorporated herein by reference intheir entireties.

The panoramic or surround view or top view or bird's eye view visionsystem of the present invention provides a stitched image that isderived from images or image data captured by a plurality of cameras atthe vehicle (such as the forward facing camera, the rearward facingcamera, the left side camera and the right side camera). In order toprovide a substantially seamless stitched image as displayed on a videodisplay screen viewable by the driver when normally operating theequipped vehicle, respective adjoining image portions of the capturedimages displayed as a composite image need to be properly aligned, sothat, for example, a road marking or parking line or similar feature orcharacteristic that is at least partially captured by one of the camerasand thus is displayed in its respective image portion, and that is alsoat least partially captured by a second one of the cameras and thus isdisplayed in its respective image portion, is properly aligned anddisplayed on the video screen as it should appear when the cameras areproperly calibrated and as it would appear in real life on the ground ator near the vehicle. By means of the present invention, misalignments,discontinuities and improperly joined lines in the stitched compositeimage that may be caused by the likes of vehicle loading and/or roadsurface out-of-flatness or the like, can be mitigated and preferably atleast substantially eliminated.

Uneven road conditions may make the stitched image look broken andmisaligned even with properly calibrated cameras. For example, one ormore wheels of the vehicle may be raised higher than the other wheels ofthe vehicle due to the road camber or tilt. Also, a change in vehicledynamics that differ from the vehicle dynamics at the time of theinitial multi-camera system calibration (when the cameras and visionsystem were initially calibrated, typically at the automobilemanufacturing plant itself) may make the stitched/composite displayedimage appear misaligned at the stitching edges/joints. Such a change invehicle dynamics may be due to uneven or changed load distribution atthe vehicle, such as a heavy load added at one side or at the rear ofthe vehicle or at the top of the vehicle causing the vehicle to tilttowards one side or rearward, or due to uneven passenger loading or thelike or uneven vehicle suspension performance or the like. Thus, thepresent invention provides a dynamic multi-camera top view or bird's eyeview or surround view system, which is operable to correct for suchrun-time/real-time errors or misalignments in the stitched images thatwould otherwise occur, and is operable to so do on an image-by-image orframe-by-frame basis in real time while the vehicle is being normallyoperated.

As described in U.S. patent application Ser. No. 13/847,815, filed Mar.20, 2013, and published on Oct. 31, 2013 as U.S. Publication No.US-2013-0286193, which is hereby incorporated herein by reference in itsentirety, a multi camera image processing system may be operable todetect the elevation of objects by its sheering effect when the objectis seen in an overlapping region by at least two of the vision systemfish eye lens cameras at different angles. The dynamic multi-cameravision system of the present invention also uses the overlapping regionsof the fields of view of the respective cameras of the system, such as,for example, the overlapping fields of view of a driver-side camera atan exterior sideview mirror of the vehicle and a front camera (with theoverlapping region being, for example, at the front left or driver-sidecorner of the vehicle). The system is operable to process image datarepresentative of overlapping regions in the adjacent camera fields ofview to detect corresponding features. When such corresponding featuresare detected or determined, the system may match the features.

For example, and with reference to FIGS. 5 and 6, the right side orpassenger side camera captures an image sideward of the vehicle, withthe captured image encompassing an area rearward and sideward of thevehicle (FIG. 5), while the rear camera captures an image rearward ofthe vehicle, with the captured image encompassing an area rearward andsideward of the vehicle (FIG. 6). The system is operable to detectcorresponding features in the imaged scene, such as the points shown inFIGS. 5 and 6, and may match the features so that, when the images arestitched, the matched corresponding features are generally orsubstantially aligned and displayed appropriately.

The system of the present invention is operable to generate affinetransforms or other suitable transforms that minimize the error infeature mapping from different views. Such an affine transformation oraffine map is a transformation that typically preserves straight lines(such that all points lying on a line initially still lie on a lineafter transformation) and that typically preserves ratios of distancesbetween points lying on a straight line and that typically preservessets of parallel lines so that they remain generally parallel aftertransformation.

The system of the present invention is further operable to apply thetransforms to correct the final stitched image. For example, the systemmay determine when there is a change in the characteristics detected ordetermined in the overlapping regions of the fields of view and mayadapt or adjust the image stitching algorithm or process to accommodatefor the detected or determined change. Because the system is a dynamicsystem, the system is operable to re-generate the transforms for everyfew frames to correct for dynamic conditions (such as changes in theroad conditions or changes in the vehicle loading or the like). Thepresent invention thus is operable to adapt the stitching of the imagesafter the cameras are calibrated to adapt the system and accommodatechanges in the vehicle and/or road surface that may otherwise adverselyaffect the quality or seamlessness of the stitched images. The presentinvention thus provides a vision system that maintains a substantiallyseamless stitched image for displaying for view by the driver of thevehicle through various driving and/or loading conditions of thevehicle.

The system thus determines differences in features present inoverlapping image data captured by a front (or rear) camera and inoverlapping image data captured by a side camera, and may adjust theimage stitching algorithm accordingly, such that the displayed stitchedimage provides an enhanced display (such as by assuring that image datacaptured by the front camera and image data captured by the side cameraappropriately align in the display viewed by the driver of the equippedvehicle so that there is no discontinuity or incongruity in the imagebeing displayed). A feature can be any one of a curb at the side of theroad being traveled by the equipped vehicle, a lane boundary marking atthe side of the road being traveled by the equipped vehicle, an edgeregion of the road being traveled by the equipped vehicle, a mailboxbeing approached and passed by the equipped vehicle, a pole or post orpillar at the side of the road being traveled by the equipped vehicle,and/or the like.

For example, while traveling forward on a road, the equipped vehicle mayapproach a mailbox located at or near a curb to the left of the equippedvehicle. The vehicle is equipped with a forward facing camera mounted atthe front of the vehicle and a side-mounted camera mounted at the leftor driver-side exterior rearview mirror of the vehicle. As the mailboxis being approached by the vehicle, it is first viewed/imaged by thefront or forward facing camera but is not yet viewed/imaged by the sidemounted camera. As the vehicle approaches closer to the mailbox, themailbox is viewed/imaged by both the forward facing camera and theside-mounted camera, and is located at a region or area where the fieldsof view of both cameras overlap and encompass the mailbox. In this fieldof view overlap region, the same object or feature (for example, themailbox) is commonly viewed by the forward facing camera and the sidemounted camera. However, the front camera's view/image of the object orfeature will be different from that of the side camera's view/image ofthe object or feature. In accordance with the present invention, thisdifference is determined by the image processor and the image stitchingalgorithm is adjusted according to the determined difference.

For example, if the mailbox is located in the overlap region of twocameras (such as at or near or along a stitching line or region of thecaptured images), the stitching algorithm is adjusted so that themailbox images (as captured by, for example, both the front camera andthe side camera) generally correspond to one another to provide agenerally smooth transition of the mailbox image as the mailbox displayimage moves or transitions from the front camera image to theoverlapping region and to the side camera image. For example, the imagestitching algorithm may be adjusted to adjust an orientation of themailbox so that the generally vertical post or structure of the mailboxin images derived from the image data captured by the front camera isgenerally aligned with the generally vertical post or structure of themailbox in images derived from the image data captured by the sidecamera. The system thus accommodates for misalignment of characteristicsof features or objects between the images of the front camera and sidecamera (or the rear camera and side camera).

Therefore, the present invention provides a dynamic process forautomatically stitching images captured by the cameras of a multi-cameravision system of a vehicle, such as for a top-down or bird's eye viewdisplay for viewing by the driver of the vehicle. For example, thepresent invention provides a system that adjusts the stitching ofadjacent images captured by the front, rear and driver side andpassenger side cameras for a surround view or top-down view visionsystem of a vehicle.

The present invention utilizes the fact that an object that is locatedin the overlapping region of the fields of view of two of the cameras ofthe multi-camera system (such as, for example, a driver-side camera anda rear camera of the vehicle) is commonly imaged by that side camera andthat rear camera. However, because of the different and known differentplacement or positions of the cameras at the equipped vehicle, theobject will be viewed differently by the respective first and secondcamera. Such differences are accommodated for and are factored in viathe initial factory calibration so that the system displays the objectappropriately and correctly in the stitched composite image. However,if, for example, during dynamic loading of the vehicle, these respectiveviews differ from what should occur in an unloaded or unperturbedideally calibrated scenario, this determined change or difference can beutilized using image processing/algorithmic techniques to adjust theimage stitching to compensate for the dynamic loading of the vehicle inreal time. Thus, utilizing the present invention, the stitching of thecaptured images (to provide or generate the stitched composite image)may be adapted for vehicle load changes and level changes and the likein real time and during normal operation of the vehicle.

Optionally, when the vehicle is in motion (such as driving forwardly orrearwardly), the ground level calibration may be checked permanently orepisodically or from time to time, or preferably at times when thesystem may have unused processing resources. As shown in FIGS. 8-11,image portions 14 a′, 14 b′, 14 c′ and 14 d′ taken by the respectivecameras 14 a, 14 b, 14 c and 14 d, and are used to create to a stitchedvirtual top view image. In FIG. 8, objects 30 are shown symbolized byboxes passing the scene surrounding the vehicle when the vehicle isdriving forward. A stitched virtual top view image may be derived fromthe image portions 14 a′, 14 b′, 14 c′ and 14 d′. When the vehicle isleveled normally, the objects 30 are supposed to pass the scene ingenerally straight lines when the vehicle is driving straight, shown inFIG. 9 with a picked out test object 31 highlighted in black. FIG. 10shows the identical scene as shown in FIG. 9, with two test objects 31highlighted and other objects erased (for clearer illustration), showingthat, when the vehicle is leveled normally, the test objects 31 aresupposed to pass the scene in generally straight lines sideward or leftand right from the vehicle when the vehicle is driving straight. FIG. 11shows a scene similar to these in FIGS. 9 and 10, with two test objects31 highlighted, with the vehicle driving straight as in FIGS. 9 and 10,but with the vehicle rolled by an angle ψ to the right, when the rollingangle ψ is uncompensated by the vision system, the passing paths of thetracked (and also the untracked) objects do not follow straight linesand the stitching thresholds 33 do not match any more.

The system of the present invention may track objects (such as objects30 and tracked objects 31 shown in FIGS. 8-11) detected coming into thedetection range in front of the vehicle (or to the rear of the vehiclewhen the vehicle is backing up) and may continue to track these objectswhen the vehicle is passing the objects and the objects are entering theviewing areas of different cameras (14 a to 14 d) of the vehicle, suchas can be seen with reference to FIGS. 8 and 9. The passing way of thevehicle is known and consistent relative to immobile objects. When anobject 31 (tracked) is entering the viewing area (14 d′ or 14 c′) of aside camera (14 d or 14 c), with the object being earlier tracked by thefront camera 14 b (while passing), the entering point (32 in FIG. 10)and passing way of the object through the side camera image isexpectable or predictable.

When there is an offset 33 or a different angle, the side camera will beoff by a different angle which points to mis-calibration of the camerasrelative to each other (in roll, yaw, pitch angle or transversaldisplacement) or the vehicle itself is skewed. Typically, both cases canbe separated or distinguished by the fact that skewing of the vehiclemay happen temporarily while a camera displacement stays in allconditions. Preferably, the system of the present invention may becontrolled in a way that first the camera alignment relative to anothercamera is calibrated and from then on skewing of the vehicle may beassumed when object path inconsistencies appear. Another decisioncriteria may be that object path inconsistencies may appear (in major)at all four cameras at the same time (such as shown in FIG. 11) during acamera displacement of one camera occurrences inconsistencies withinthree stitched camera images. When comparing FIG. 10, which shows thecase of a leveled vehicle, to FIG. 11, which shows the case of a skewedvehicle (rolled to the right in x axis), the passing ways of objectspassing the vehicle on the left side have an offset into the samedirection (to the right) as objects passing the vehicle at the rightside. Responsive to determination of such skewing or rolling of thevehicle, the system may alter the virtual ground plane in a way that avirtual top view as like in FIG. 10 results.

Additionally or optionally, the system may also take an input from anenvironmental sensor 28 (beside the cameras) into account. If not doneby the vision computing system alone, the system may utilize distancemeasuring sensors, such as, for example, ultrasound (exemplary case inFIGS. 7A and 7B), Time of Flight (TOF), RADAR, LIDAR, LADAR and/or thelike, for detecting the true ground plane level and elevations 25 nearby the vehicle 10 (the detection range which may have (still) adequateresolution may be from about 0.2 m up to about 10 m depending on thesensors type (equates to ultrasound cloud 29 in FIGS. 7A and 7B)) forfeeding it into the vision system to correct the virtual (assumed)ground plane for the virtual top view vision projection in the mannerthe vehicle is rolled or pitched. The system either may detect changesfor adding/subtracting delta tunings or absolute values for plain input.

For example, and as shown in FIGS. 7A and 7B, the vehicle 10 may includean ultrasound distance measuring system having an ultrasound sensor 28integrated into a side mirror housing 27 of the vehicle 10. Theultrasound sensor 28 is shown with an effective ultrasound expansioncloud illustration 29 (not actually visible), and shown with the vehiclein a horizontal level. If the vehicle is rolled or tilted to its rightby an angle ψ, the measuring system, which rolls with the vehicle,detects a closer distance to objects and the ground. FIG. 7B shows avehicle turned by an angle ψ (rolled in x axis) in comparison to aleveled car in FIG. 7A. The system may detect the ground or at leastelevated objects on the ground 25 are coming closer to the mirrorintegrated sensor due to that rolling of the vehicle by the angle ψ. Thesensor data may come as a depth image or in a point cloud which may needto be combined with the image giving (camera) system image data bysensor fusion. The ground plane, objects (with elevation) and horizonmay be distinguishable by processing of sensor data.

As an additional or alternative way to detect that the vehicle is notfully leveled horizontally (rolled in x axis) and/or has a pitch angle(in y axis) or the like, the system may utilize the compression ratio ofthe shock absorber system or suspension system of all wheels shockabsorbers of the vehicle. Nowadays vehicles usually have sensors todetect the compression rate and usually this signal is provided via thevehicle's CAN or vehicle bus network, which makes it possible that thevision system is able to process this information. By such processing ofsuspension information (for the individual wheels of the vehicle) thesystem may determine uneven suspension of the corner regions of thevehicle and may correct the virtual (assumed) ground plane for thevirtual top view vision projection in accordance with the determinedratio that the vehicle is rolled or pitched (skewed).

As a preferred embodiment, the system of the present invention maydetermine the ground level calibration roughly on hand of the shockabsorber sensor or suspension data (via CAN) and may use the opticalcalibration referred to above as a successive fine tuning procedure,which may use the optional range sensor data in addition or in backup.

It is desirable for vision overlays to look quite artificial when thesebecome mapped over a real scene view captured by automotive cameras.Earlier filings, such as International Publication No. WO 2013/086249,which is hereby incorporated herein by reference in its entirety, usevisual effects such as ‘motion blurring’ and/or partially transparencyin dependence of the viewing distance and direction of the virtualcamera to help this fact. Nevertheless there is room for improvement.

As soon as the 3D shape or topography of the world within a displayedscene (in an automotive vision system) is known (using a mono camera and3D recognition by ‘structure from motion’ or ‘structure from focallength’ or ‘depths estimation from focal blur’ (known from known TV 2Dto 3D conversion algorithms (TP-Vision in Philips TV) or by using stereocameras (including infrared) or using a mono cam era and additionally asecond or non-vision-based sensor system, such as LIDAR, RADAR, TOF orthe like), especially the ground surface structure the car is restingon, will be renderable. The overlays of the present invention may bemapped or placed (or calculated or controlled or adjusted) responsive tothe world's topographical structure. Such overlays may utilize aspectsof the overlays described in International Publication No. WO2009036176, which is hereby incorporated herein by reference in itsentirety.

Overlays related to the subject or equipped or host vehicle and overlaysdedicated to other vehicles, infrastructure or points of interest (POI)may be mapped onto the structures' topography as like projected ordropped from over top. By that the overlays cover the mapping structurefolding around all ups and downs which strengthens the three dimensionalimpression of the scenery (see FIG. 12 (flat overlay)), since the top ofthe overlay may follow to the topography.

The overlays may have a three dimensional structure which top elevationmay be responsive to the ground plane (virtually). By that the top ofthe overlay repeats the topography of its ground plane (See FIG. 13(solid overlay)).

The two and three dimensional overlays may be fully or partiallytransparent for preventing anything important in the scene is hidden byan overlay (and not detected or as important classified). See FIG. 14(transparent overlay).

FIGS. 12, 13 and 14 have in common a scene of a pedestrian side walkalong a road. The displayed view, which may be displayed at a vehicle'scompartment display when the vehicle is reversing into a long sideparking spot, is indicated by the box or outline. As common, thecurvature of the reversing path according to the steering wheel angle orthe automated path planning system is indicated by the line of(positions of) overlays. The overlays may have different colorsaccording to the distance to the vehicle's rear. The overlays may havedifferent colors or may change color or blink or flash or may alter intransparency or in brightness or may be animated, such as by torsion,wobbling, vibrating or jumping (such as in a manner like jelly or thelike), responsive to the distance to the vehicle's rear, the speed atwhich possible colliding hazards or possible damage threatening objectsmay close to the vehicle (or that the vehicle may close to the objects),or the remaining (eventually estimated, by the trajectory in past) timeuntil closing to an object or the like. In all three of FIGS. 12-14, theexamples of the vehicle's driving path lead or progress substantially ormassively up the curb at the side of the road and over the side walk.The curbs may be visible by the step in heights of the overlay's top andoptionally additionally by a thread indication or highlighting asdescribed above (such as by a change in color, blinking, brightness oranimation or the like).

With reference now to FIG. 15, the system of the present invention mayvisualize that the arrangement of the overlays may reflect the threedimensional topography of a scene. The shown perspective views of afreeway situation, such as may be displayed in a rearview mirrorreplacement camera display (such as by utilizing aspects of the systemsdescribed in U.S. provisional application Ser. No. 61/935,057, filedFeb. 3, 2014, which has now been filed as U.S. patent application Ser.No. 14/898,493, filed Dec. 15, 2015, and published on May 19, 2016 asU.S. Publication No. US-2016-0137126), may be used for visualizing ordisplaying the projected path of a vehicle driving behind the subject orequipped vehicle but in another lane. Due to the overlayed boxes (whichmay have different highlightings as described above) following theprospective projected path of the following vehicle, the equippedvehicle's driver may be able to readily capture at which specific lanethe following vehicle is approaching.

Not shown in the drawings is that when the overlays infringe (take thesame space) another (real or virtual) object, the overlays may appear ina faded color or in dot or dashed or phantom lines. Optionally, theobjects may pile on top of each other (when running across a wall forexample). Optionally, the overlays may shrink in size or step slightlyto the side when logically possible. For example, an inserted ‘Point ofInterest Label’ may give way to a parking overlay which position issystematically crucial.

It is also desirable to gather and provide context depending labeling ofinformation in vehicle vision systems. Storing a lot of information onboard the vehicle takes a lot of storage space and it may becomeoutdated comparably fast. Solutions are known to provide usefulinformation to the car aid systems and to the driver via wireless radiochannels from and to a dedicated server or from the world wide web, suchas by utilizing aspects of the systems described in U.S. patentapplication Ser. No. 13/660,306, filed Oct. 25, 2012, now U.S. Pat. No.9,146,898, which is hereby incorporated herein by reference in itsentirety. New standards are coming up as like HTML5. This is forproviding displaying contents. To maintain the displaying contents up todate adds costs to the system. These costs may be fully or partiallyborne by third party providers. These may finance their services byadding commercial contents to the ‘useful contents’. For example, theseproviders may employ a service for ‘points of interest’ (POI) which maycome up as virtual labels added to real camera taken scenes of an urbanstreet or environment. The providers may collect money from stores andthe company or store logos or labels may be presented in a preferred orselected manner (such as in the foreground, brighter, bigger, moreexposed, blinking, or the like).

The vision system of the present invention may employ a spam filter forincoming foreign viewing contents. The spam filter properties may beconfigurable. The configuration may come from third parties or may bedone by the vision system provider or by the vehicle OEM or by thevehicle owner itself (over an HMI) or from a remote device or system(such as, for example, a cell phone app or the like). The spam filterproperties may be context dependent (situational, position dependent,vehicle state dependent, vehicle occupation or occupant dependent,driving state dependent). For example there may be contents preferred bythe adult occupants, while other or additional contents may becomedisplayed only when a teenager is also on board the vehicle. Forexample, the teenager may love fast food restaurants while the parentsprefer exquisite restaurants. Thus, the vision display system may showPOIs of all types of restaurants when the children are in the vehiclewith their parents, but will show just some exquisite flaggedrestaurants when the parents are alone in the vehicle. The “flagging”may come from the vision system provider, by the vehicle OEM or by thevehicle owner itself (over an HMI) or from remote (such as, for example,a cell phone app). Besides that all other annoying content may beoptionally suppressible by a spam filter.

Optionally, there may be different spam filter property data sets thatthe vehicle users are able to store in the vehicle data storage on theircell phone or remote. It may be possible that the spam filter properties(and all other vision system settings) may stay with the driver whichever vehicle he or she uses. The data set may be downloaded from aremote location (server) via any data channel or transferred by his/hercell phone.

The vision system may collect POIs by itself such as, for example, byemploying an object detection algorithm on known labels or by requestingthe background content according to detected maybe unknown labels. Thefound POIs may be distributed in a database common to all or tocertified users.

Optionally, the vision system (utilizing the forward facing camera and arearward facing camera and other cameras disposed at the vehicle withexterior fields of view) may be part of or may provide a display of atop-down view or birds-eye view system of the vehicle or a surround viewat the vehicle, such as by utilizing aspects of the vision systemsdescribed in International Publication Nos. WO 2010/099416; WO2011/028686; WO 2012/075250; WO 2013/019795; WO 2012/075250; WO2012/145822; WO 2013/081985; WO 2013/086249 and/or WO 2013/109869,and/or U.S. patent application Ser. No. 13/333,337, filed Dec. 21, 2011,now U.S. Pat. No. 9,264,672, which are hereby incorporated herein byreference in their entireties.

The system includes an image processor operable to process image datacaptured by the camera or cameras, such as for detecting objects orother vehicles or pedestrians or the like in the field of view of one ormore of the cameras. For example, the image processor may comprise anEYEQ2 or EYEQ3 image processing chip available from Mobileye VisionTechnologies Ltd. of Jerusalem, Israel, and may include object detectionsoftware (such as the types described in U.S. Pat. Nos. 7,855,755;7,720,580 and/or 7,038,577, which are hereby incorporated herein byreference in their entireties), and may analyze image data to detectvehicles and/or other objects. Responsive to such image processing, andwhen an object or other vehicle is detected, the system may generate analert to the driver of the vehicle and/or may generate an overlay at thedisplayed image to highlight or enhance display of the detected objector vehicle, in order to enhance the driver's awareness of the detectedobject or vehicle or hazardous condition during a driving maneuver ofthe equipped vehicle.

The vehicle may include any type of sensor or sensors, such as imagingsensors or radar sensors or lidar sensors or ladar sensors or ultrasonicsensors or the like. The imaging sensor or camera may capture image datafor image processing and may comprise any suitable camera or sensingdevice, such as, for example, an array of a plurality of photosensorelements arranged in at least 640 columns and 480 rows (preferably amegapixel imaging array or the like), with a respective lens focusingimages onto respective portions of the array. The photosensor array maycomprise a plurality of photosensor elements arranged in a photosensorarray having rows and columns. The logic and control circuit of theimaging sensor may function in any known manner, and the imageprocessing and algorithmic processing may comprise any suitable meansfor processing the images and/or image data.

For example, the vision system and/or processing and/or camera and/orcircuitry may utilize aspects described in U.S. Pat. Nos. 7,005,974;5,760,962; 5,877,897; 5,796,094; 5,949,331; 6,222,447; 6,302,545;6,396,397; 6,498,620; 6,523,964; 6,611,202; 6,201,642; 6,690,268;6,717,610; 6,757,109; 6,802,617; 6,806,452; 6,822,563; 6,891,563;6,946,978; 7,859,565; 5,550,677; 5,670,935; 6,636,258; 7,145,519;7,161,616; 7,230,640; 7,248,283; 7,295,229; 7,301,466; 7,592,928;7,881,496; 7,720,580; 7,038,577; 6,882,287; 5,929,786 and/or 5,786,772,and/or International Publication Nos. WO 2011/028686; WO 2010/099416; WO2012/061567; WO 2012/068331; WO 2012/075250; WO 2012/103193; WO2012/0116043; WO 2012/0145313; WO 2012/0145501; WO 2012/145818; WO2012/145822; WO 2012/158167; WO 2012/075250; WO 2012/0116043; WO2012/0145501;; WO 2012/154919; WO 2013/019707; WO 2013/016409; WO2013/019795; WO 2013/067083; WO 2013/070539; WO 2013/043661; WO2013/048994; WO 2013/063014, WO 2013/081984; WO 2013/081985; WO2013/074604; WO 2013/086249; WO 2013/103548; WO 2013/109869; WO2013/123161; WO 2013/126715; WO 2013/043661 and/or WO 2013/158592 and/orU.S. patent applications, Ser. No. 14/163,325, filed Jan. 24, 2014, andpublished on Jul. 31, 2014 as U.S. Publication No. US-2014-0211009; Ser.No. 14/159,772, filed Jan. 21, 2014, now U.S. Pat. No. 9,068,390; Ser.No. 14/107,624, filed Dec. 16, 2013, now U.S. Pat. No. 9,140,789; Ser.No. 14/102,981, filed Dec. 11, 2013, now U.S. Pat. No. 9,558,409; Ser.No. 14/102,980, filed Dec. 11, 2013, and published Jun. 19, 2014 as U.S.Publication No. US-2014-0168437; Ser. No. 14/098,817, filed Dec. 6,2013, and published Jun. 19, 2014 as U.S. Publication No.US-2014-0168415; Ser. No. 14/097,581, filed Dec. 5, 2013, now U.S. Pat.No. 9,481,301; Ser. No. 14/093,981, filed Dec. 2, 2013, now U.S. Pat.No. 8,917,169; Ser. No. 14/093,980, filed Dec. 2, 2013, now U.S. Pat.No. 10,025,994; Ser. No. 14/082,573, filed Nov. 18, 2013, now U.S. Pat.No. 9,743,002; Ser. No. 14/082,574, filed Nov. 18, 2013, now U.S. Pat.No. 9,307,640; Ser. No. 14/082,575, filed Nov. 18, 2013, now U.S. Pat.No. 9,090,234; Ser. No. 14/082,577, filed Nov. 18, 2013, now U.S. Pat.No. 8,818,042; Ser. No. 14/071,086, filed Nov. 4, 2013, now U.S. Pat.No. 8,886,401; Ser. No. 14/076,524, filed Nov. 11, 2013, now U.S. Pat.No. 9,077,962; Ser. No. 14/052,945, filed Oct. 14, 2013, now U.S. Pat.No. 9,707,896; Ser. No. 14/046,174, filed Oct. 4, 2013, now U.S. Pat.No. 9,707,896; Ser. No. 14/016,790, filed Oct. 3, 2013, now U.S. Pat.No. 9,761,142; Ser. No. 14/036,723, filed Sep. 25, 2013, now U.S. Pat.No. 9,446,713; Ser. No. 14/016,790, filed Sep. 3, 2013, now U.S. Pat.No. 9,761,142; Ser. No. 14/001,272, filed Aug. 23, 2013, now U.S. Pat.No. 9,233,641; Ser. No. 13/970,868, filed Aug. 20, 2013, now U.S. Pat.No. 9,365,162; Ser. No. 13/964,134, filed Aug. 12, 2013, now U.S. Pat.No. 9,340,227; Ser. No. 13/942,758, filed Jul. 16, 2013, and publishedon Jan. 23, 2014 as U.S. Publication No. US-2014-0025240; Ser. No.13/942,753, filed Jul. 16, 2013, and published Jan. 30, 2014 as U.S.Publication No. US-2014-0028852; Ser. No. 13/927,680, filed Jun. 26,2013, and published Jan. 2, 2014 as U.S. Publication No.US-2014-0005907; Ser. No. 13/916,051, filed Jun. 12, 2013, now U.S. Pat.No. 9,077,098; Ser. No. 13/894,870, filed May 15, 2013, now U.S. Pat.No. 10,089,537; Ser. No. 13/887,724, filed May 6, 2013, now U.S. Pat.No. 9,670,895; Ser. No. 13/852,190, filed Mar. 28, 2013, and publishedAug. 29, 2013 as U.S. Publication No. US-2013-0222593; Ser. No.13/851,378, filed Mar. 27, 2013, now U.S. Pat. No. 9,319,637; Ser. No.13/848,796, filed Mar. 22, 2012, and published Oct. 24, 2013 as U.S.Publication No. US-2013-0278769; Ser. No. 13/847,815, filed Mar. 20,2013, and published Oct. 31, 2013 as U.S. Publication No.US-2013-0286193; Ser. No. 13/800,697, filed Mar. 13, 2013, and publishedOct. 3, 2013 as U.S. Publication No. US-2013-0258077; Ser. No.13/785,099, filed Mar. 5, 2013, now U.S. Pat. No. 9,565,342; Ser. No.13/779,881, filed Feb. 28, 2013, now U.S. Pat. No. 8,694,224; Ser. No.13/774,317, filed Feb. 22, 2013, now U.S. Pat. No. 9,269,263; Ser. No.13/774,315, filed Feb. 22, 2013, and published Aug. 22, 2013 as U.S.Publication No. US-2013-0215271; Ser. No. 13/681,963, filed Nov. 20,2012, now U.S. Pat. No. 9,264,673; Ser. No. 13/660,306, filed Oct. 25,2012, now U.S. Pat. No. 9,146,898; Ser. No. 13/653,577, filed Oct. 17,2012, now U.S. Pat. No. 9,174,574; and/or Ser. No. 13/534,657, filedJun. 27, 2012, and published Jan. 3, 2013 as U.S. Publication No.US-2013-0002873, and/or U.S. provisional applications, Ser. 61/931,811,filed Jan. 27, 2014; Ser. No. 61/919,129, filed Dec. 20, 2013; Ser. No.61/919,130, filed Dec. 20, 2013; Ser. No. 61/919,131, filed Dec. 20,2013; Ser. No. 61/919,147, filed Dec. 20, 2013; Ser. No. 61/919,138,filed Dec. 20, 2013, Ser. No. 61/919,133, filed Dec. 20, 2013; Ser. No.61/918,290, filed Dec. 19, 2013; Ser. No. 61/915,218, filed Dec. 12,2013; Ser. No. 61/912,146, filed Dec. 5, 2013; Ser. No. 61/911,666,filed Dec. 4, 2013; Ser. No. 61/911,665, filed Dec. 4, 2013; Ser. No.61/905,461, filed Nov. 18, 2013; Ser. No. 61/905,462, filed Nov. 18,2013; Ser. No. 61/901,127, filed Nov. 7, 2013; Ser. No. 61/895,610,filed Oct. 25, 2013; Ser. No. 61/895,609, filed Oct. 25, 2013; Ser. No.61/893,489, filed Oct. 21, 2013; Ser. No. 61/886,883, filed Oct. 4,2013; Ser. No. 61/879,837, filed Sep. 19, 2013; Ser. No. 61/879,835,filed Sep. 19, 2013; Ser. No. 61/878,877, filed Sep. 17, 2013; Ser. No.61/875,351, filed Sep. 9, 2013; Ser. No. 61/869,195, filed. Aug. 23,2013; Ser. No. 61/864,835, filed Aug. 12, 2013; Ser. No. 61/864,836,filed Aug. 12, 2013; Ser. No. 61/864,837, filed Aug. 12, 2013; Ser. No.61/864,838, filed Aug. 12, 2013; Ser. No. 61/856,843, filed Jul. 22,2013, Ser. No. 61/845,061, filed Jul. 11, 2013; Ser. No. 61/844,630,filed Jul. 10, 2013; Ser. No. 61/844,173, filed Jul. 9, 2013; Ser. No.61/844,171, filed Jul. 9, 2013; Ser. No. 61/842,644, filed Jul. 3, 2013;Ser. No. 61/840,542, filed Jun. 28, 2013; Ser. No. 61/838,619, filedJun. 24, 2013; Ser. No. 61/838,621, filed Jun. 24, 2013; Ser. No.61/837,955, filed Jun. 21, 2013; Ser. No. 61/836,900, filed Jun. 19,2013; Ser. No. 61/836,380, filed Jun. 18, 2013; Ser. No. 61/834,129,filed Jun. 12, 2013; Ser. No. 61/833,080, filed Jun. 10, 2013; Ser. No.61/830,375, filed Jun. 3, 2013; Ser. No. 61/830,377, filed Jun. 3, 2013;Ser. No. 61/825,752, filed May 21, 2013; Ser. No. 61/825,753, filed May21, 2013; Ser. No. 61/823,648, filed May 15, 2013; Ser. No. 61/823,644,filed May 15, 2013; Ser. No. 61/821,922, filed May 10, 2013; Ser. No.61/819,835, filed May 6, 2013; Ser. No. 61/819,033, filed May 3, 2013;Ser. No. 61/816,956, filed Apr. 29, 2013; Ser. No. 61/815,044, filedApr. 23, 2013; Ser. No. 61/814,533, filed Apr. 22, 2013; Ser. No.61/813,361, filed Apr. 18, 2013; Ser. No. 61/810,407, filed Apr. 10,2013; Ser. No. 61/808,930, filed Apr. 5, 2013; Ser. No. 61/806,674,filed Mar. 29, 2013; Ser. No. 61/793,592, filed Mar. 15, 2013; Ser. No.61/772,015, filed Mar. 4, 2013; Ser. No. 61/772,014, filed Mar. 4, 2013;Ser. No. 61/770,051, filed Feb. 27, 2013; Ser. No. 61/766,883, filedFeb. 20, 2013; Ser. No. 61/760,366, filed Feb. 4, 2013, and/or Ser. No.61/760,364, filed Feb. 4, 2013, which are all hereby incorporated hereinby reference in their entireties. The system may communicate with othercommunication systems via any suitable means, such as by utilizingaspects of the systems described in International Publication Nos. WO2010/144900; WO 2013/043661 and/or WO 2013/081985, and/or U.S. patentapplication Ser. No. 13/202,005, filed Aug. 17, 2011, now U.S. Pat. No.9,126,525, which are hereby incorporated herein by reference in theirentireties.

The imaging device and control and image processor and any associatedillumination source, if applicable, may comprise any suitablecomponents, and may utilize aspects of the cameras and vision systemsdescribed in U.S. Pat. Nos. 5,550,677; 5,877,897; 6,498,620; 5,670,935;5,796,094; 6,396,397; 6,806,452; 6,690,268; 7,005,974; 7,937,667;7,123,168; 7,004,606; 6,946,978; 7,038,577; 6,353,392; 6,320,176;6,313,454 and 6,824,281, and/or International Publication Nos. WO2010/099416; WO 2011/028686 and/or WO 2013/016409, and/or U.S. Pat.Publication No. US 2010-0020170, and/or U.S. patent application Ser. No.13/534,657, filed Jun. 27, 2012, and published Jan. 3, 2013 as U.S.Publication No. US-2013-0002873, which are all hereby incorporatedherein by reference in their entireties. The camera or cameras maycomprise any suitable cameras or imaging sensors or camera modules, andmay utilize aspects of the cameras or sensors described in U.S. Pat.Publication No. US-2009-0244361 and/or U.S. patent application Ser. No.13/260,400, filed Sep. 26, 2011, now U.S. Pat. No. 8,542,451, and/orU.S. Pat. Nos. 7,965,336 and/or 7,480,149, which are hereby incorporatedherein by reference in their entireties. The imaging array sensor maycomprise any suitable sensor, and may utilize various imaging sensors orimaging array sensors or cameras or the like, such as a CMOS imagingarray sensor, a CCD sensor or other sensors or the like, such as thetypes described in U.S. Pat. Nos. 5,550,677; 5,670,935; 5,760,962;5,715,093; 5,877,897; 6,922,292; 6,757,109; 6,717,610; 6,590,719;6,201,642; 6,498,620; 5,796,094; 6,097,023; 6,320,176; 6,559,435;6,831,261; 6,806,452; 6,396,397; 6,822,563; 6,946,978; 7,339,149;7,038,577; 7,004,606; 7,720,580 and/or 7,965,336, and/or InternationalPublication Nos. WO 2009/036176 and/or WO 2009/046268, which are allhereby incorporated herein by reference in their entireties.

The camera module and circuit chip or board and imaging sensor may beimplemented and operated in connection with various vehicularvision-based systems, and/or may be operable utilizing the principles ofsuch other vehicular systems, such as a vehicle headlamp control system,such as the type disclosed in U.S. Pat. Nos. 5,796,094; 6,097,023;6,320,176; 6,559,435; 6,831,261; 7,004,606; 7,339,149 and/or 7,526,103,which are all hereby incorporated herein by reference in theirentireties, a rain sensor, such as the types disclosed in commonlyassigned U.S. Pat. Nos. 6,353,392; 6,313,454; 6,320,176 and/or7,480,149, which are hereby incorporated herein by reference in theirentireties, a vehicle vision system, such as a forwardly, sidewardly orrearwardly directed vehicle vision system utilizing principles disclosedin U.S. Pat. Nos. 5,550,677; 5,670,935; 5,760,962; 5,877,897; 5,949,331;6,222,447; 6,302,545; 6,396,397; 6,498,620; 6,523,964; 6,611,202;6,201,642; 6,690,268; 6,717,610; 6,757,109; 6,802,617; 6,806,452;6,822,563; 6,891,563; 6,946,978 and/or 7,859,565, which are all herebyincorporated herein by reference in their entireties, a trailer hitchingaid or tow check system, such as the type disclosed in U.S. Pat. No.7,005,974, which is hereby incorporated herein by reference in itsentirety, a reverse or sideward imaging system, such as for a lanechange assistance system or lane departure warning system or for a blindspot or object detection system, such as imaging or detection systems ofthe types disclosed in U.S. Pat. Nos. 7,720,580; 7,038,577; 5,929,786and/or 5,786,772, and/or U.S. patent applications, Ser. No. 11/239,980,filed Sep. 30, 2005, now U.S. Pat. No. 7,881,496, and/or U.S.provisional applications, Ser. No. 60/628,709, filed Nov. 17, 2004; Ser.No. 60/614,644, filed Sep. 30, 2004; Ser. No. 60/618,686, filed Oct. 14,2004; Ser. No. 60/638,687, filed Dec. 23, 2004, which are herebyincorporated herein by reference in their entireties, a video device forinternal cabin surveillance and/or video telephone function, such asdisclosed in U.S. Pat. Nos. 5,760,962; 5,877,897; 6,690,268 and/or7,370,983, and/or U.S. Publication No. US-2006-0050018, which are herebyincorporated herein by reference in their entireties, a traffic signrecognition system, a system for determining a distance to a leading ortrailing vehicle or object, such as a system utilizing the principlesdisclosed in U.S. Pat. Nos. 6,396,397 and/or 7,123,168, which are herebyincorporated herein by reference in their entireties, and/or the like.

Optionally, the circuit board or chip may include circuitry for theimaging array sensor and or other electronic accessories or features,such as by utilizing compass-on-a-chip or EC driver-on-a-chip technologyand aspects such as described in U.S. Pat. No. 7,255,451 and/or U.S.Pat. No. 7,480,149; and/or U.S. patent applications, Ser. No.11/226,628, filed Sep. 14, 2005 and published Mar. 23, 2006 as U.S.Publication No. US-2006-0061008, and/or Ser. No. 12/578,732, filed Oct.14, 2009, now U.S. Pat. No. 9,487,144, which are hereby incorporatedherein by reference in their entireties.

Optionally, the vision system may include a display for displayingimages captured by one or more of the imaging sensors for viewing by thedriver of the vehicle while the driver is normally operating thevehicle. Optionally, for example, the vision system may include a videodisplay device disposed at or in the interior rearview mirror assemblyof the vehicle, such as by utilizing aspects of the video mirror displaysystems described in U.S. Pat. No. 6,690,268 and/or U.S. patentapplication Ser. No. 13/333,337, filed Dec. 21, 2011, now U.S. Pat. No.9,264,672, which are hereby incorporated herein by reference in theirentireties. The video mirror display may comprise any suitable devicesand systems and optionally may utilize aspects of the compass displaysystems described in U.S. Pat. Nos. 7,370,983; 7,329,013; 7,308,341;7,289,037; 7,249,860; 7,004,593; 4,546,551; 5,699,044; 4,953,305;5,576,687; 5,632,092; 5,677,851; 5,708,410; 5,737,226; 5,802,727;5,878,370; 6,087,953; 6,173,508; 6,222,460; 6,513,252 and/or 6,642,851,and/or European patent application, published Oct. 11, 2000 underPublication No. EP 0 1043566, and/or U.S. patent application Ser. No.11/226,628, filed Sep. 14, 2005 and published Mar. 23, 2006 as U.S.Publication No. US-2006-0061008, which are all hereby incorporatedherein by reference in their entireties. Optionally, the video mirrordisplay screen or device may be operable to display images captured by arearward viewing camera of the vehicle during a reversing maneuver ofthe vehicle (such as responsive to the vehicle gear actuator beingplaced in a reverse gear position or the like) to assist the driver inbacking up the vehicle, and optionally may be operable to display thecompass heading or directional heading character or icon when thevehicle is not undertaking a reversing maneuver, such as when thevehicle is being driven in a forward direction along a road (such as byutilizing aspects of the display system described in PCT Application No.PCT/US2011/056295, filed Oct. 14, 2011 and published Apr. 19, 2012 asInternational Publication No. WO 2012/051500, which is herebyincorporated herein by reference in its entirety).

Optionally, a video mirror display may be disposed rearward of andbehind the reflective element assembly and may comprise a display suchas the types disclosed in U.S. Pat. Nos. 5,530,240; 6,329,925;7,855,755; 7,626,749; 7,581,859; 7,446,650; 7,370,983; 7,338,177;7,274,501; 7,255,451; 7,195,381; 7,184,190; 5,668,663; 5,724,187 and/or6,690,268, and/or in U.S. patent applications, Ser. No. 12/091,525,filed Apr. 25, 2008, now U.S. Pat. No. 7,855,755; Ser. No. 11/226,628,filed Sep. 14, 2005 and published Mar. 23, 2006 as U.S. Publication No.US-2006-0061008; and/or Ser. No. 10/538,724, filed Jun. 13, 2005 andpublished Mar. 9, 2006 as U.S. Publication No. US-2006-0050018, whichare all hereby incorporated herein by reference in their entireties. Thedisplay is viewable through the reflective element when the display isactivated to display information. The display element may be any type ofdisplay element, such as a vacuum fluorescent (VF) display element, alight emitting diode (LED) display element, such as an organic lightemitting diode (OLED) or an inorganic light emitting diode, anelectroluminescent (EL) display element, a liquid crystal display (LCD)element, a video screen display element or backlit thin film transistor(TFT) display element or the like, and may be operable to displayvarious information (as discrete characters, icons or the like, or in amulti-pixel manner) to the driver of the vehicle, such as passenger sideinflatable restraint (PSIR) information, tire pressure status, and/orthe like. The mirror assembly and/or display may utilize aspectsdescribed in U.S. Pat. Nos. 7,184,190; 7,255,451; 7,446,924 and/or7,338,177, which are all hereby incorporated herein by reference intheir entireties. The thicknesses and materials of the coatings on thesubstrates of the reflective element may be selected to provide adesired color or tint to the mirror reflective element, such as a bluecolored reflector, such as is known in the art and such as described inU.S. Pat. Nos. 5,910,854; 6,420,036 and/or 7,274,501, which are herebyincorporated herein by reference in their entireties.

Changes and modifications in the specifically described embodiments maybe carried out without departing from the principles of the presentinvention, which is intended to be limited only by the scope of theappended claims as interpreted according to the principles of patentlaw.

1. A method for stitching image data captured by multiple vehicularcameras, the method comprising: equipping a vehicle with a vehicularvision system comprising a control and a plurality of cameras disposedat the vehicle so as to have respective fields of view exterior thevehicle; wherein the control comprises a data processor; wherein theplurality of cameras comprises (i) a rear camera disposed at a rearportion of the vehicle and having a field of view at least rearward ofthe vehicle, (ii) a left-side camera disposed at a left side portion ofthe vehicle and having a field of view at least sideward of the vehicle,(iii) a right-side camera disposed at a right side portion of thevehicle and having a field of view at least sideward of the vehicle and(iv) a front camera disposed at a front portion of the vehicle andhaving a field of view at least forward of the vehicle; wherein thefield of view of the left-side camera partially overlaps the field ofview of the front camera and partially overlaps the field of view of therear camera; wherein the field of view of the right-side camerapartially overlaps the field of view of the front camera and partiallyoverlaps the field of view of the rear camera; capturing image data bythe plurality of cameras and providing captured image data to the dataprocessor of the control; processing at the control the providedcaptured image data; detecting, via processing at the control of theprovided captured image data, an object present in an overlappingportion of the fields of view of first and second cameras of theplurality of cameras; determining, via processing at the control of theprovided captured image data, a feature of the detected object presentin the overlapping portion of the fields of view of the first and secondcameras; tracking, via processing at the control of the providedcaptured image data, the determined feature of the detected object as itmoves within the overlapping portion of the fields of view of the firstcamera and the second camera; stitching, via processing at the controlof the provided captured image data, image data captured by the firstand second cameras to form stitched images; determining differencebetween (i) the determined feature of the detected object as captured bythe first camera and as tracked as the determined feature of thedetected object moves within the overlapping portion of the field ofview of the first camera and (ii) the determined feature of the detectedobject as captured by the second camera and as tracked as the determinedfeature of the detected object moves within the overlapping portion ofthe field of view of the second camera; and adjusting stitching ofcaptured image data responsive to the determined difference between (i)the determined feature of the detected object as captured by the firstcamera and as tracked as the determined feature of the detected objectmoves within the overlapping portion of the field of view of the firstcamera and (ii) the determined feature of the detected object ascaptured by the second camera and as tracked as the determined featureof the detected object moves within the overlapping portion of the fieldof view of the second camera.
 2. The method of claim 1, comprisingdetermining, via processing at the control of the provided capturedimage data, a change in pitch or roll of the vehicle, and adjustingstitching of captured image data responsive to the determined change inpitch or roll of the vehicle.
 3. The method of claim 2, whereindetermining the change in pitch or roll of the vehicle comprisesdetermining, via processing at the control of the provided capturedimage data, a change in distance to the detected object exterior thevehicle.
 4. The method of claim 2, wherein the change in pitch or rollof the vehicle is determined at least in part responsive to a suspensionsystem of the vehicle.
 5. The method of claim 1, wherein stitching ofcaptured image data forms seamless stitched images.
 6. The method ofclaim 1, wherein the first camera comprises the front camera or the rearcamera, and wherein the second camera comprises the left-side camera orthe right-side camera.
 7. The method of claim 1, comprising determining,via processing at the control of the provided captured image data,differences in the determined feature of the detected object overmultiple frames of captured image data.
 8. The method of claim 1,wherein the determined feature of the detected object comprises avertical feature of the detected object.
 9. The method of claim 1,wherein the determined feature of the detected object comprises ahorizontal feature of the detected object.
 10. The method of claim 1,wherein the detected object comprises at least one selected from thegroup consisting of (i) a curb at a side of a road on which the vehicleis being driven, (ii) a mailbox being approached and passed by thevehicle and (iii) a post at a side of a road on which the vehicle isbeing driven.
 11. The method of claim 1, wherein adjusting stitching ofcaptured image data is at least in part responsive to a non-image-basedsensor of the vehicle.
 12. The method of claim 11, comprisingdetermining, via processing of sensor data captured by thenon-image-based sensor, a distance to the detected object exterior thevehicle, and determining, via processing of sensor data captured by thenon-image-based sensor, a change in distance to the detected object thatis indicative of a change in pitch or roll of the vehicle.
 13. A methodfor stitching image data captured by multiple vehicular cameras, themethod comprising: equipping a vehicle with a vehicular vision systemcomprising a control and a plurality of cameras disposed at the vehicleso as to have respective fields of view exterior the vehicle; whereinthe control comprises a data processor; wherein the plurality of camerascomprises (i) a rear camera disposed at a rear portion of the vehicleand having a field of view at least rearward of the vehicle, (ii) aleft-side camera disposed at a left side portion of the vehicle andhaving a field of view at least sideward of the vehicle, (iii) aright-side camera disposed at a right side portion of the vehicle andhaving a field of view at least sideward of the vehicle and (iv) a frontcamera disposed at a front portion of the vehicle and having a field ofview at least forward of the vehicle; wherein the field of view of theleft-side camera partially overlaps the field of view of the frontcamera and partially overlaps the field of view of the rear camera;wherein the field of view of the right-side camera partially overlapsthe field of view of the front camera and partially overlaps the fieldof view of the rear camera; capturing image data by the plurality ofcameras and providing captured image data to the data processor of thecontrol; processing at the control the provided captured image data;detecting, via processing at the control of the provided captured imagedata, an object present in an overlapping portion of the fields of viewof first and second cameras of the plurality of cameras; wherein thefirst camera comprises the front camera or the rear camera, and whereinthe second camera comprises the left-side camera or the right-sidecamera; determining, via processing at the control of the providedcaptured image data, a feature of the detected object present in theoverlapping portion of the fields of view of the first and secondcameras; tracking, via processing at the control of the providedcaptured image data, the determined feature of the detected object as itmoves within the overlapping portion of the fields of view of the firstcamera and the second camera; stitching, via processing at the controlof the provided captured image data, image data captured by the firstand second cameras to form stitched images; determining change in pitchor roll of the vehicle by determining difference between (i) thedetermined feature of the detected object as captured by the firstcamera and as tracked as the determined feature of the detected objectmoves within the overlapping portion of the field of view of the firstcamera and (ii) the determined feature of the detected object ascaptured by the second camera and as tracked as the determined featureof the detected object moves within the overlapping portion of the fieldof view of the second camera; and adjusting stitching of captured imagedata responsive to the determined change in pitch or yaw of the vehicle.14. The method of claim 13, wherein determining the change in pitch orroll of the vehicle in part comprises determining, via processing at thecontrol of the provided captured image data, a change in distance to thedetected object exterior the vehicle.
 15. The method of claim 13,wherein determining the change in pitch or roll of the vehicle is inpart responsive to a suspension system of the vehicle.
 16. The method ofclaim 13, comprising determining, via processing at the control of theprovided captured image data, differences in the determined feature ofthe detected object over multiple frames of captured image data.
 17. Themethod of claim 13, wherein the determined feature of the detectedobject comprises a vertical feature of the detected object.
 18. Themethod of claim 13, wherein the determined feature of the detectedobject comprises a horizontal feature of the detected object.
 19. Amethod for stitching image data captured by multiple vehicular cameras,the method comprising: equipping a vehicle with a vehicular visionsystem comprising a control and a plurality of cameras disposed at thevehicle so as to have respective fields of view exterior the vehicle;wherein the control comprises a data processor; wherein the plurality ofcameras comprises (i) a rear camera disposed at a rear portion of thevehicle and having a field of view at least rearward of the vehicle,(ii) a left-side camera disposed at a left side portion of the vehicleand having a field of view at least sideward of the vehicle, (iii) aright-side camera disposed at a right side portion of the vehicle andhaving a field of view at least sideward of the vehicle and (iv) a frontcamera disposed at a front portion of the vehicle and having a field ofview at least forward of the vehicle; wherein the field of view of theleft-side camera partially overlaps the field of view of the frontcamera and partially overlaps the field of view of the rear camera;wherein the field of view of the right-side camera partially overlapsthe field of view of the front camera and partially overlaps the fieldof view of the rear camera; capturing image data by the plurality ofcameras and providing captured image data to the data processor of thecontrol; processing at the control the provided captured image data;detecting, via processing at the control of the provided captured imagedata, an object present in an overlapping portion of the fields of viewof first and second cameras of the plurality of cameras; wherein thefirst camera comprises the front camera or the rear camera, and whereinthe second camera comprises the left-side camera or the right-sidecamera; wherein the detected object comprises at least one selected fromthe group consisting of (i) a curb at a side of a road on which thevehicle is being driven, (ii) a mailbox being approached and passed bythe vehicle and (iii) a post at a side of a road on which the vehicle isbeing driven; determining, via processing at the control of the providedcaptured image data, a feature of the detected object present in theoverlapping portion of the fields of view of the first and secondcameras; tracking, via processing at the control of the providedcaptured image data, the determined feature of the detected object as itmoves within the overlapping portion of the fields of view of the firstcamera and the second camera; stitching, via processing at the controlof the provided captured image data, image data captured by the firstand second cameras to form stitched images; determining differencebetween (i) the determined feature of the detected object as captured bythe first camera and as tracked as the determined feature of thedetected object moves within the overlapping portion of the field ofview of the first camera and (ii) the determined feature of the detectedobject as captured by the second camera and as tracked as the determinedfeature of the detected object moves within the overlapping portion ofthe field of view of the second camera; and adjusting stitching ofcaptured image data responsive to the determined difference between (i)the determined feature of the detected object as captured by the firstcamera and as tracked as the determined feature of the detected objectmoves within the overlapping portion of the field of view of the firstcamera and (ii) the determined feature of the detected object ascaptured by the second camera and as tracked as the determined featureof the detected object moves within the overlapping portion of the fieldof view of the second camera.
 20. The method of claim 19, comprisingdetermining, via processing at the control of the provided capturedimage data, a change in pitch or roll of the vehicle, and adjustingstitching of captured image data responsive to the determined change inpitch or roll of the vehicle.
 21. The method of claim 20, whereindetermining the change in pitch or roll of the vehicle comprisesdetermining, via processing at the control of the provided capturedimage data, a change in distance to the detected object exterior thevehicle.
 22. The method of claim 20, wherein the change in pitch or rollof the vehicle is determined at least in part responsive to a suspensionsystem of the vehicle.
 23. The method of claim 19, wherein thedetermined feature of the detected object comprises a vertical featureof the detected object.
 24. The method of claim 19, wherein thedetermined feature of the detected object comprises a horizontal featureof the detected object.